1. Field of the Invention
The present invention relates to a function to track a specified object in an image.
2. Description of the Related Art
Some digital cameras, video cameras, or monitoring camera systems in which a camera is communicatively connected to a computer have a function to track a specified object in an image by continuously detecting the specified object in a movie.
Japanese Patent Application Laid-Open No. 04-170870, for example, discusses a configuration in which an object to be tracked is specified from a frame image, a luminance signal in the area of the object is stored and an area most similar to the stored luminance signal is extracted from another frame image, thereby tracking the object.
FIGS. 7A and 7B are schematic diagrams illustrating a method for tacking an object using a luminance signal in the area of the object. FIG. 7A illustrates a frame image in a movie captured by a digital camera. Suppose that the face of a person who looks to the right is specified as an object to be tracked. A tracking frame indicating the object to be tracked is displayed on the monitor of a digital camera with the tracking frame superposed on the image to surround the face of the person.
The digital camera stores each of pixels of luminance signals in an object area to be tracked surrounded by the tacking frame. Thereafter, in the frame image, a comparison area equal in size to the object area to be tracked is set, and the sum total of differences of luminance signals between the pixels in the object area to be tracked and in the comparison area is obtained. The position of the comparison area is sequentially shifted and the sum total of differences of luminance signals in each position is obtained to detect the position of the comparison area where the sum total of differences is the smallest.
In the other frame image, the area having a pattern (shape) of the luminance signal most similar to a pattern of the luminance signal in the object area to be tracked in a frame image is determined as an area where the same object as an object to be tracked exists and the tacking frame is updated.
FIG. 7B illustrates the state of update. In FIG. 7B, the position of the tacking frame is updated so that the luminance signal in the tacking frame exists in the area having the pattern of the luminance signal most correlative to that of the luminance signal in the area in the tacking frame in FIG. 7A. The stored pattern of the luminance signal is updated to a pattern of the luminance signal in a newly set tracking frame. Such an operation is repeated to track a specified object in a movie.
FIGS. 8A to 8C are schematic diagrams illustrating issues to be solved in a method for tracking an object using the luminance signal of an object area.
In a frame image illustrated in FIG. 8A, the face of a person is specified as an object to be tracked. In a frame image illustrated in FIG. 8B, the face of the person as an object to be tracked is behind a tree as an obstacle. In the frame image illustrated in FIG. 8B, if an area most correlative to the pattern of the luminance signal in the area in the tracking frame illustrated in FIG. 8A is detected and set as a new tracking frame, a part of the face and a part of the tree being the obstacle are included in the area of a new tracking frame. In a frame image illustrated in FIG. 8C, an area where a part of the tree being the obstacle exists is selected as an area highly correlative to the pattern of the luminance signal in the tracking frame. In subsequent frame images, the tree may be continuously tracked.
To avoid this, even in the area having the pattern of the luminance signal most correlative to that of the stored luminance signal in the area in the tacking frame, if the correlation is not greater than a predetermined value, it is determined that an object to be tracked is lost and tracking is stopped. In other words, the area having the pattern of the luminance signal most correlative to the pattern of the luminance signal in the tracking frame of the frame image illustrated in FIG. 8A is detected in the frame image illustrated in FIG. 8B. However, since it is determined that the correlation does not reach the predetermined value, the tracking frame is deleted in the frame image illustrated in FIG. 8C. Thereby, if a different object may be set as an object to be tracked en route, it is possible to stop tracking.
Such a process for stopping tracking causes another issue. When a tracking is performed using a correlation between the patterns of luminance signals, a change in the direction of an object or in the attitude of the object temporarily and substantially changes the correlation between the patterns of luminance signals.
FIGS. 9A to 9D illustrate such a change described above. FIGS. 9A to 9D are schematic diagrams illustrating another issue to be solved in a method for tracking an object using a luminance signal in the area of the object.
In a frame image illustrated in FIG. 9A, the face of a person is specified as an object to be tracked. In a frame image illustrated in FIG. 9B, the person turns to the other side to conceal the face of the person who looks to the right, which causes only the back of the head of the person to appear. In a frame image illustrated in FIG. 9C, the person turns to the left to conceal the back of the head, which causes the face of the person who looks to the left to appear. In the frame images illustrated in FIGS. 9A to 9C, the person as an object to be tracked merely changes its direction. However, a correlation between the patterns of the luminance signals does not reach a predetermined value during that time, so that the tracking frame may be deleted in the frame images as illustrated in FIGS. 9C and 9D.
Thus, it is difficult to find whether the reason the correlation between the luminance signals is lowered is attributed to the tracking of a different object or a mere change in the direction of an object to be tracked.
It is also difficult in the case where a tracking is performed based on the correlation between the patterns of the luminance signals using a color signal such as a chroma signal or a hue signal instead of a luminance signal.